Intrusion Detection in Resource-Constrained Cyber Networks: A Restless Multi-Armed Bandit Approach

We consider a large-scale cyber network with N components. Each component is either in a healthy state (0) or an abnormal state (1). Due to intrusions, the state of each component transits from 0 to 1 over time according to an arbitrary stochastic process. At each time, a subset of K (K < N) components are probed and those observed in abnormal states are fixed. The objective is to design a dynamic probing strategy that minimizes the long-term network cost incurred at all abnormal components. We formulate the problem as a Restless Multi-Armed Bandit (RMAB) process. We show that this class of RMAB is indexable and Whittle index can be obtained in closedform. For homogeneous networks, we show that Whittle index policy achieves the optimal performance with a simple structure that does not require any prior knowledge on the intrusion processes. These results also apply to scheduling in multi-class queuing networks with single buffers.